OPEN PEER COMMENTARY Dwelling on the Past

With the growing number of fraudulent and nonr replicability of experiments performed in laboratories world practices that investigators may be using to increase the replicability. We laud them for thoughtful intentions and ex domains: the structure of psychological science and the generalizability. The former represents a methodological/sta opportunity. Copyright © 2013 John Wiley & Sons, Ltd. Copyright © 2013 John Wiley & Sons, Ltd. confirmation and may very well be fictional. To resolve this situation, we need to dwell on the past, and several courses of action present themselves. First, psychology requires thorough examination, for example by an American Psychological Association taskforce, to propose a list of psychological findings that feature at the textbook level but in fact are still in need of direct replication. In a second step, those findings that are in need of replication can be reinvestigated in research that implements the proposals of Asendorpf et al. The work initiated by the Open Science Framework (http://openscienceframework.org/) has gone a long way in constructing a methodology to guide massive replication efforts and can be taken as a blueprint for this kind of work. Psychology needs to improve its research methodology, and the procedures proposed by Asendorpf et al. will undoubtedly contribute to that goal. However, psychology also cannot avoid the obligation to look back and to find out which studies are textbook-proof and which are not. By implementing sensible procedures to further the veracity of our empirical work, psychologists have the opportunity to lead by example, an opportunity that we cannot afford to miss. Scientific Advances in eplicable reports in psychological science, many question the wide. The focus of Asendorpf and colleagues is on research likelihood of publication while unknowingly undermining tend their recommendations by focusing on two additional need to distinguish between minimal replicability and tistical problem, whereas the latter represents a theoretical Although cases of outright fraud are rare and not unique to psychology, psychological science has been rocked in the past few years by a few cases of failed replications and fraudulent science. Among practices suggested by Asendorpf et al. as contributing to these outcomes are data selection and formulating decisions about sample size on the basis of statistical significance rather than statistical power. We laud Asendorpf et al. for their thoughtful and timely recommendations and hope their paper becomes required reading. We focus here on two domains they did not address: the structure of psychological science and the need to distinguish between minimal replicability and generalizability. Publication of a new scientific finding should be viewed more as a promissory note than a final accounting. Science is not a solitary pursuit; it is a social process. If a scientific finding cannot be independently verified, then it cannot be regarded as an empirical fact. Minimal replicability, defined as an empirical finding that can be repeated by an independent investigator using the same operationalizations, situations, and time points in an independent sample of participants, is the currency of science. Asendorpf et al. distinguish among reproducibility (duplication by an independent investigator analysing the same dataset), replicability (observation with other random samples), and generalizability (absence of dependence on an originally unmeasured variable). Issues of replicability and generalizability have been addressed before in psychology. Basic psychological research, with its emphasis on experimental control, was once criticized for yielding statistically reliable but trivial effects (e.g., Appley, 1990; Staats, 1989). Allport (1968) decades ago noted that scientific gains result from this hard-nosed approach, but he lamented the lack of generalizing power of many neat and elegant experiments: ‘It is for this reason that some current investigations seem to end up in elegantly polished triviality—snippits of empiricism, but nothing more’ (p. 68). Many psychological phenomena, ranging from attention to racism, are multiply determined (Schachter, Eur. J. Pers. 27: 120–144 (2013)